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AUTOMATICALLY DESIGNED THREE-DIMENSIONAL GRAPHICAL 
ENVIRONMENTS FOR INFORMATION DISCOVERY AND 
VISUALIZATION 

FIELD OF THE INVENTION 
[0001] The present invention relates to a technique for generating a three-dimensional 
graphical environment and in particular, this disclosure provides an apparatus and 
method for automatically generating three-dimensional graphical environments for 
information discovery and visualization dependent on a selected viewing scheme. 

BACKGROUND OF THE INVENTION 

[0002] Databases are often used to store information relating to large numbers of 
objects such as products, works of art, museum pieces, or books. Due to their size, it 
can be difficult to search or browse through the information stored in these databases. 
There are usually specialized interfaces for visualizing the information depending on 
the application. However, there are some universally desirable features of the 
interface regardless of the application. It should enable the user to navigate and 
browse through the database quickly. It should be visually compelling and interactive 
to allow the user to effortlessly manipulate the visualization. Finally, it should be an 
intuitive interface. 

[0003] A particularly important application of information visualization is in the area 
of shopping. People interested in shopping used to have only the option of going to 
their local brick-and-mortar stores to shop. When the Internet revolution occurred, it 
was thought that online shopping could provide the shopper with much better ways to 
visualize and browse through a store's inventory. Online shopping did not end up 
overtaking traditional brick-and-mortar stores as expected simply because it could not 
replace many of the inherent benefits of regular shopping. Traditional stores are 
professionally designed and have very pleasant layouts. A visually immersive 
experience and natural way of navigating makes shopping in these stores very 
appealing. In many cases, shoppers do not have a specific item in mind and may want 
to simply "window shop" through the products. In fact, shoppers quite often discover 
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items they wish to purchase purely by accident; the brick-and-mortar stores are very 
conducive for browsing and this type of "accidental discovery." 

[0004] Current online shopping is certainly not without its merits, and consequently, 
many retail stores have created an online presence as well. Online shopping can be 
quickly accomplished in the comfort of one's own home or office at any time, day or 
night. Online stores can provide a much wider selection and larger inventory since 
they require zero physical store space; this in turn generally leads to lower prices. 
Current online stores often include a web-based user interface that allows the user to 
"browse" through lists or images of products by category. Specifically, if a user 
selects a category using the interface, either a written list of products or images 
representing the products are displayed that fall within the selected category. 
Alternatively, the user interface can include a search engine that allows the user to 
perform a word or term search to locate a specific product or category within the 
database. Such a search engine is very efficient when the shopper knows precisely the 
model of the object to be purchased. 

[0005] There are many problems associated with the above browsing and searching 
techniques. For instance, if a user does not know what category the product that they 
are looking for falls into, they must either guess which category to look through or 
look through each list of products for each category until the desired product is found. 
In addition, the user may not know the exact terminology used to describe the desired 
product on the website and as a result, the term/word search engine can become 
useless. As a result, due to the size of the database of products and the manner in 
which the user interface is designed, it becomes a cumbersome task requiring the user 
to navigate through continuous windows of web pages of product information. In the 
case of accidental discovery, the above described search tools are not even applicable 
and current browsing techniques become inadequate or cumbersome. 

[0006] Users encounter a different browsing experience when viewing images, such 
as artwork, on a website. In this case, the user is typically not searching for a specific 
item but instead is simply randomly viewing the objects (i.e., artwork) of the database. 
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As a result, the above browsing and searching techniques are particularly unsuited for 
this type of viewing experience since the user is not searching for any specific object. 

[0007] A much more compelling interface is the use of interactive three-dimensional 
graphical environments, which combine many of the benefits of online shopping with 
those of brick-and-mortar stores. Three-dimensional graphical (or virtual) 
environments have been used in the past to allow viewing of and navigation through 
information in an intuitive manner. Specifically, a three-dimensional graphical 
environment allows a user to intuitively navigate within a virtual space and hence 
easily locate and view items within the space, thus overcoming the disadvantages of 
browsing and searching by category and search engines. Even the most novice 
computer user can appreciate its immersive and natural interface. 

O [0008] Many three-dimensional graphical environments, especially those found in the 

O 

fj gaming and entertainment industries, possess the "look and feel" of a real-world 

!T environment, and in fact can be quite visually stunning. However, these environments 

fy 

fl j are predesigned and predetermined for their specific application. As a result, the 

user's experience is restricted to the designer's preconceived ideas as to how to best 
|f arrange the environment for optimized navigation, searching, browsing, and accidental 
yj discovery. If the user's and designer's views are different, then the user's experience 

5; within the environment may be diminished. Moreover, such static environments 
H cannot easily accommodate dynamically changing data objects and user 

customization. Hence, what would be desirable is a customizable and automatically 
designed three-dimensional graphical environment based on user preferences. 

[0009] The present invention is an apparatus and method of automatically generating a 
three-dimensional graphical environment for viewing a plurality of objects in a 
database having improved browsability. 
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SUMMARY OF THE INVENTION 

[0010] An apparatus and method for browsing a plurality of data objects represented 
by images displayed within a three-dimensional graphical environment is described. 
A plurality of data object arrangement schemes are provided for arranging 
representative images of the data objects within the three-dimensional graphical 
environment dependent on a selected one of the schemes. The data objects are then 
arranged within the three-dimensional graphical environment according to the selected 
scheme. Schemes can be user selected such that by selecting a new scheme, the 
objects are rearranged according to the newly selected scheme. In one embodiment, 
the environment is partitioned into three-dimensional areas and representative images 
of the data objects are arranged within the partitioned environment such that 
representative images of related groups of data objects with respect to the selected 
scheme are displayed within the same areas. 

[0011] In one embodiment, an apparatus for browsing a plurality of stored data objects 
within a displayed three-dimensional graphical environment includes a plurality of 
selectable data object arrangement schemes and a three-dimensional graphics 
processing unit. The graphics processing unit generates three-dimensional image data 
corresponding to the environment and representative images of the data objects such 
that the representative images are arranged within the displayed environment 
dependent on a selected one of the schemes. 

[0012] In another embodiment, the graphics processing unit includes two processing 
units: first, a layout processor for determining the layout of the environment and the 
placement of the representative images of the objects within the environment; and 
second, an environment creation processor for determining the appearance of the 
environment and representative images within the environment. The layout processor 
includes an environment partitioner for generating environment partition information 
corresponding to partitioned three-dimensional areas within the environment and an 
object placement locator for generating information corresponding to the location of 
related objects within each partitioned area. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0013] Fig. 1 illustrates a prior art system for rendering and displaying a three- 
dimensional graphical environment; 

[0014] Fig. 2 illustrates a first embodiment of an apparatus for generating a three- 
dimensional graphical environment according to the present invention; 

[0015] Fig. 3 illustrates a first embodiment of a method for generating a three- 
dimensional graphical environment; 

[0016] Fig. 4 illustrates a second embodiment of a method for generating a three- 
p dimensional graphical environment; and 

M 

[0017] Figs. 5A - 5D illustrate examples of three-dimensional graphical 
environments embodied according to the apparatus and method of the present 
invention. 
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DETAILED DESCRIPTION 

[0018] An apparatus and method for browsing representative images of data objects 
within a displayed three-dimensional graphical environment is described in which 
images are arranged within the environment according to a selected one of a plurality 
of data object arrangement schemes. According to this description, a data object is 
broadly defined as data stored within a database that corresponds to a tangible or 
intangible object or element. Examples of types of data objects can include but are 
not limited to image data, audio data, word processing data, video data, and 3-D 
models. Examples of types of objects or elements can include but are not limited to 
movie videos, CDs, works of art, electronic word processing documents, electronic 
personnel records, and commercial inventory. If, for instance, the object type is movie 
videos, a specific object of that type would be a particular movie. 



Vf: [0019] Each data object has associated data referred to as metadata which corresponds 

4* to any data other than the data object itself that describes or relates to the data object, 

% 

1^ and each data object may have different types of metadata. For instance, in the case in 

J;* which the object type is movie videos, types of metadata may include data 

C3 corresponding to the title of the movie, director of the movie, the release date of the 

Zl movie, the actors of the movie, genre of movie (e.g., comedy, drama) etc. In contrast, 

in the case in which the object type is artwork, types of metadata may include title of 
the work and school of art (e.g. modern, impressionistic). Metadata can be stored with 
the data object or may be stored separately such that a link exists between the data 
object and its associated metadata. 

[0020] In general, a plurality of data objects stored within a database are related such 
that they can be logically viewed, categorized, grouped and/or arranged in some way 
with respect to their associated data object or metadata. For instance, movies stored 
within a database can be grouped according to director so as to allow a viewer to 
review the data by a particular director. 
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[0021] According to the present invention, representative images of data objects 
stored within a database are displayed within a three-dimensional graphical 
environment to provide a user who wants to browse the database the ability to 
intuitively view the data objects. In accordance with the present invention, 
representative images can be embodied as text labels, natural images, graphical 
images, document and file folder images, video images, three-dimensional models, or 
any type of visual data. Moreover, representative images can be interactive when 
selected by the user. For instance, a representative image embodied as a three- 
dimensional model may be rotated by a user to provide the user with additional views 
of the model. In addition, representative images may be selected to access additional 
related multi-media data. For instance, a two-dimensional representative image may 

j, 4 be selected so as to access an audio clip associated with the data object, access a three- 

dimensional model of the corresponding two-dimensional image, access a video 

fy sequence, etc. 

:;n 3 

Jj^ [0022] An example of a system for rendering and displaying a three-dimensional 

■•F image is shown in Fig. 1 that includes a graphics processing unit 10, a rendering 

| ; * engine 1 1, a user output 12, a user input 13, and memory 14. The graphics processing 

unit 10 functions to receive data object data 10 A and generate three-dimensional 

to 

O imaging data 10B corresponding to a three-dimensional graphical environment 

Tl including representative images of the data objects within the three-dimensional 

environment as is well known in the field of graphics imaging. Rendering Engine 1 1 
receives imaging data 10B and generates rendering data 11A for driving user output 
12. For instance, rendering data 1 1 A drives a monitor (not shown) to display the 
three-dimensional graphical environment including the representative images. User 
input 13 provides an interface between a user and the three-dimensional environment 
so as to allow the user to interact with the three-dimensional environment. For 
instance, the user input 13 allows the user to change a point of view within the three- 
dimensional environment by moving a displayed cursor using a keyboard or mouse. 
Other types of user input 13 include but are not limited to a joystick and a pressure 
sensitive screen. Graphics processing unit 10 may function to generate the 
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representative images of the data objects or the representative image data may be 
stored within the memory 14 and linked to the data object database. 

[0023] According to one embodiment of the present invention, graphics processing 
unit 10 is implemented according to the apparatus 20 shown in Fig. 2 including a 
layout processor 21 and an environment creation processor 22. The layout processor 
21 functions to generate three-dimensional imaging information 21C defining, at least, 
the partitioning information of the three-dimensional graphical environment and 
placement information defining the placement of representative images corresponding 
to a plurality of data objects within the partitioned three-dimensional environment. 
Both of the partitioning information and the placement information are dependent on 
at least a selected one of a plurality of data object arrangement schemes. Data object 
selection is determined by the data object scheme selection control signal coupled to 
the layout processor 21. Selection of the type of arrangement scheme determines the 
manner in which the environment is partitioned by layout processor 21 and determines 
the placement of representative data objects within the partitioned areas. According to 
HF the present invention, different arrangement schemes correspond to ways of relating 

data objects to one another within a database so as to group the data objects dependent 
on their relations. In one embodiment, metadata is used to relate objects so as to 
O group the data objects. 



W 
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[0024] In one embodiment, the partitioning and placement within the layout is also 
dependent on a set of environment design rules (Fig. 2) that define the placement of 
any object within the environment. For example, to create a partitioned area within 
the three-dimensional graphical environment, an object (such as a wall) is used. In 
this case, an environment design rule would, for example, ensure that walls/objects are 
not too close or overlapping, determine how or whether to insert corridors within the 
environment to allow passage from one partitioned area to another, ensure that 
representative images of data objects are not too close/overlapping, reduce the 
occurrence of long skinny rooms in preference for squarer rooms, etc. In another 
example, the environment design rules can specify the locations of particular 
departments in a store (e.g. the men's department is always in the northwest corner, or 
the produce section of a virtual grocery store is always on the west side, etc). 
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Similarly, the environment design rules can ensure that a virtual store is laid out in a 
similar manner as a user selected brick-and-mortar store at a specific location. 

[0025] In another embodiment, the selected data object arrangement scheme is based 
on at least one metadata type according to which the data objects are to be grouped 
and displayed. Once the metadata type is identified, the environment partitioner 21 A 
analyzes the metadata for all data objects to identify data objects having the same 
metadata for the identified metadata type. Data objects identified as having the same 
metadata are grouped together. The number of data objects in each group is 
determined and the size of each partitioned area required to accommodate each group 
of objects is determined. From the size information for each of the partitioned areas, 
the environment partitioner 21 A generates coordinate information within the three- 
dimensional graphical environment. The coordinate information is provided to the 
object placement locator 21B. The object placement locator 21B then determines the 
placement of the grouped data objects within each area dependent on the environment 
design rules. 

[0026] Layout information 21C is coupled to the environment creation processor 22. 
Environment creation processor 22 uses the partition information including coordinate 
information and placement information to generate three-dimensional image data 
representing the partitions that are located according to the coordinate information 
provided by environment partitioner 21 A. The appearance of the three-dimensional 
graphical environment is determined by appearance design rules (Fig. 2) that 
determine the appearance of the three-dimensional environment and how the objects 
and text within the environment are to appear dependent on a selected motif or theme. 
For instance, if a database of movie videos is being displayed within a three- 
dimensional environment, and the appearance rules are set such that the motif of a 
video store is selected, then the environment creation processor may generate three- 
dimensional image data corresponding to a video store having many rooms and 
including other objects in the environment that might be identified with a video store. 
For instance, the three-dimensional video store environment may include a check out 
desk. In addition, the environment creation processor may generate image data 
corresponding to walls in the video store for partitioning the video store into three- 
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dimensional areas, and data objects may be represented by a type of video box 
generally associated with video stores. Alternatively, the environment motif may be 
an old mansion, an art museum, may be indoors or outdoors. Furthermore, the 
appearance design rules also determine style of the presented architecture (e.g. gothic, 
roman, modern), color scheme, and additional fixtures (e.g. columns, gargoyles, trim). 

[0027] It should be understood that the apparatus shown in Fig. 2 can be implemented 
in any combination of hardware, software, or firmware. 

[0028] Fig. 3 shows a first embodiment of a method for browsing a plurality of data 
objects displayed within a three-dimensional graphical environment. According to 
this embodiment, a plurality of selectable data objects arrangement schemes are 
§~* provided (block 30). According to this embodiment, different arrangement schemes 

;2~ correspond to ways of relating data objects to one another within a database so as to 

:pj group the data objects dependent on their relations. In one embodiment, data objects 

fy can be grouped according to metadata type. Provided with different arrangement 

1 schemes, images representing the plurality of data objects are arranged within a 
* displayed three-dimensional environment dependent on a selected one of the 

q arrangements schemes (block 31). 

|j [0029] An example of the method shown in Fig. 3 may be seen in the case in which a 

G 

y* user is browsing a database with an interface providing the user with the option to 

select one of a plurality of arrangement schemes. For instance, if the database 
comprises data objects corresponding to museum pieces, the interface may give the 
user the option to arrange the data objects alphabetically according to piece name or 
chronologically according to the date that they were discovered or even geographically 
by country of origin. Representative images of the data objects are then displayed 
according to the selected one of the schemes. Since the scheme is selectable 
according to the method shown in Fig. 3, the method allows a user to view the data 
objects according to any of a plurality of schemes. For instance, the user can decide to 
view the objects according to a different scheme by selecting a new scheme such that 
the data objects are rearranged within the three-dimensional graphical environment 
according to the newly selected scheme. 
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[0030] Fig. 4 illustrates a second embodiment of the method for browsing a plurality 
of objects displayed within a three-dimensional graphical environment. According to 
this embodiment, a data object arrangement scheme is selected (block 40). For 
instance, a user interface may include a drop down menu that provides a user with a 
list of arrangement schemes allowing the user to select a data object arrangement 
scheme. Next, data objects are grouped according to the selected scheme (block 41). 
Hence, once the scheme is selected, the data objects are analyzed so as to group data 
objects according to the selected scheme. The environment is then partitioned 
dependent on the number of objects per grouping (block 42). Consequently, the size 
and shape of each partitioned area is dependent on the number of objects per group. 
The three-dimensional graphical environment is then rendered having representative 
p images of the grouped data objects within each area dependent on the selected scheme 

y (block 43). Hence, the partitioned environment is displayed on a monitor such that the 

:;L.J 

IP j representative images of the grouped data objects are grouped within each area 

% i dependent on the selected scheme so as to enable intuitive browsing of the data 

IU objects in the database. 

m [0031] Figs. 5A, 5B, 5C, and 5D show examples of representative images arranged 

■J"! according to different selected schemes within a three-dimensional graphical 
O environment. As shown in these figures, a plurality of data objects corresponding to 

T k movie videos are represented within a three-dimensional graphical environment by 

images corresponding to cinematic advertising photos 50 (Figs. 5A-5D) of each 
movie. In Figs. 5A and 5B, the three-dimensional environment motif is a video store 
which is divided into three-dimensional areas (e.g., 51) by store walls 52 where each 
area corresponds to a different director. Hence, in this case, data objects are grouped 
according to the arrangement scheme of director type. Note that Figs. 5A and 5B are 
different views of the same three-dimensional graphical environment. By selecting 
this scheme, the user can browse all videos by the same director within each room. 
Figs. 5C and 5D shows the same database where representative images within the 
environment are automatically arranged according to movie genre. For instance, 
dramas are grouped within the same area, comedies are grouped within another area, 
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and romances are grouped within still another area. Hence, by selecting this scheme, 
the user can browse only a particular type of movie. 

[0032] In addition, the data objects within each area can be arranged according to an 
auxiliary scheme. For instance, in the case of the example shown in Figs. 5A and 5B, 
movies by the same director in a given partitioned three-dimensional area can be 
further arranged according to the release data, the title of the movie, or some other 
scheme. 

[0033] It should be noted that schemes may be selected by the user or may be 
automatically selected dependent on the database being browsed. 

[0034] In one embodiment of the method of the present invention, when a new 
scheme is selected, the three-dimensional environment is automatically repartitioned 
and the representative images of the data objects are rearranged within the three- 
dimensional graphical environment within the three-dimensional areas dependent on 
the newly selected scheme. 

[0035]In the preceding description, numerous specific details are set forth in order to 
provide a thorough understanding of the present invention. It will be apparent, 
however, to one skilled in the art that these specific details need not be employed to 
practice the present invention. In other instances, well-known techniques have not 
been described in detail in order to avoid unnecessarily obscuring the present 
invention. 

[0036] In addition, although elements of the present invention have been described in 
conjunction with certain embodiments, it is appreciated that the invention can be 
implemented in a variety of other ways. Consequently, it is to be understood that the 
particular embodiments shown and described by way of illustration is in no way 
intended to be considered limiting. Reference to the details of these embodiments is 
not intended to limit the scope of the claims which themselves recited only those 
features regarded as essential to the invention. 
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